Neural Network Assisted Tile Size Selection

Abstract. Data locality optimization plays a significant role in reducing the execution time of many loop-intensive kernels. Loop tiling at various levels is often used to effectively exploit data locality in deep memory hierarchies. The recent development of frameworks for parametric loop tiling of user code has lead to a widening of the range of applications that could benefit from auto-tuning of tile sizes. Current model-driven approaches suffer from limitations, such as the inability to accurately model the complex interplay between multiple hardware components that affect performance. Auto-tuning libraries such as ATLAS rely on extensive empirical search for tile size optimization, which has been shown to be very effective. However, the effectiveness of such approaches for arbitrary parametrically tiled user code has not been demonstrated. We consider the problem of selecting the best tile sizes for arbitrary user-defined programs, by sampling in the full space of tile sizes. We have developed a technique to build a performance predictor associated with a specific program. Our approach uses statistical machine learning to train an artificial neural network (ANN) to predict the performance distribution of execution time for scientific kernels. We show how this search strategy significantly improves over the variability of random search. Our observations and results on various kernels also show promise for the use of ANNs in predicting the runtime behavior for variations of tiling configurations.